19 research outputs found
Graph Attention Multi-Agent Fleet Autonomy for Advanced Air Mobility
Autonomous mobility is emerging as a new mode of urban transportation for
moving cargo and passengers. However, such fleet coordination schemes face
significant challenges in scaling to accommodate fast-growing fleet sizes that
vary in their operational range, capacity, and communication capabilities. We
introduce the concept of partially observable advanced air mobility games to
coordinate a fleet of aerial vehicle agents accounting for their heterogeneity
and self-interest inherent to commercial mobility fleets. We propose a novel
heterogeneous graph attention-based encoder-decoder (HetGAT Enc-Dec) neural
network to construct a generalizable stochastic policy stemming from the inter-
and intra-agent relations within the mobility system. We train our policy by
leveraging deep multi-agent reinforcement learning, allowing decentralized
decision-making for the agents using their local observations. Through
extensive experimentation, we show that the fleets operating under the HetGAT
Enc-Dec policy outperform other state-of-the-art graph neural network-based
policies by achieving the highest fleet reward and fulfillment ratios in an
on-demand mobility network.Comment: 12 pages, 12 figures, 3 table
Uncertainty-Aware Online Merge Planning with Learned Driver Behavior
Safe and reliable autonomy solutions are a critical component of
next-generation intelligent transportation systems. Autonomous vehicles in such
systems must reason about complex and dynamic driving scenes in real time and
anticipate the behavior of nearby drivers. Human driving behavior is highly
nuanced and specific to individual traffic participants. For example, drivers
might display cooperative or non-cooperative behaviors in the presence of
merging vehicles. These behaviors must be estimated and incorporated in the
planning process for safe and efficient driving. In this work, we present a
framework for estimating the cooperation level of drivers on a freeway and plan
merging maneuvers with the drivers' latent behaviors explicitly modeled. The
latent parameter estimation problem is solved using a particle filter to
approximate the probability distribution over the cooperation level. A
partially observable Markov decision process (POMDP) that includes the latent
state estimate is solved online to extract a policy for a merging vehicle. We
evaluate our method in a high-fidelity automotive simulator against methods
that are agnostic to latent states or rely on assumptions
about actor behavior
Modeling Human Driving Behavior through Generative Adversarial Imitation Learning
Imitation learning is an approach for generating intelligent behavior when
the cost function is unknown or difficult to specify. Building upon work in
inverse reinforcement learning (IRL), Generative Adversarial Imitation Learning
(GAIL) aims to provide effective imitation even for problems with large or
continuous state and action spaces. Driver modeling is one example of a problem
where the state and action spaces are continuous. Human driving behavior is
characterized by non-linearity and stochasticity, and the underlying cost
function is unknown. As a result, learning from human driving demonstrations is
a promising approach for generating human-like driving behavior. This article
describes the use of GAIL for learning-based driver modeling. Because driver
modeling is inherently a multi-agent problem, where the interaction between
agents needs to be modeled, this paper describes a parameter-sharing extension
of GAIL called PS-GAIL to tackle multi-agent driver modeling. In addition, GAIL
is domain agnostic, making it difficult to encode specific knowledge relevant
to driving in the learning process. This paper describes Reward Augmented
Imitation Learning (RAIL), which modifies the reward signal to provide
domain-specific knowledge to the agent. Finally, human demonstrations are
dependent upon latent factors that may not be captured by GAIL. This paper
describes Burn-InfoGAIL, which allows for disentanglement of latent variability
in demonstrations. Imitation learning experiments are performed using NGSIM, a
real-world highway driving dataset. Experiments show that these modifications
to GAIL can successfully model highway driving behavior, accurately replicating
human demonstrations and generating realistic, emergent behavior in the traffic
flow arising from the interaction between driving agents.Comment: 28 pages, 8 figures. arXiv admin note: text overlap with
arXiv:1803.0104